ADAPTIVE CONTROL SYSTEM HAVING DIRECT OUTPUT FEEDBACK 
AND RELATED APPARATUSES AND METHODS " 
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Field of the Invention 

The invention is directed to a system, apparatuses and methods for adaptively 
controlhng a plant such as an aircraft, automobile, robot, or other controlled system. 
Background of the Invention 

Research in adaptive output feedback control of imcertainlionlinear dynamic 
systems is motivated by the many emerging applications that employ novel actuation 
devices for active control of flexible structures, fluid flows and combustion processes. 
These include such devices as piezoelectric fihns, and synthetic jets, which are 
typically nonlinearly coupled to the dynamics of the processes they are intended to 
control. Modeling for these applications vary fi-om having accurate low fl-equency 
models in the case of structural control problems, to having no reasonable set of 
model equations ui the case of active control of flows and combustion processes. 
Regardless of the extent of the model accuracy that may be present, an important 
aspect in any control design is the effect of parametric uncertainty and unmodeled 
dynamics. While it c^ be said the issue of parametric uncertainty is addressed within 
the context of adaptive control, very little can be said regarding robustness of the 
adaptive process to immodeled internal process dynamics. 

Synthesis approaches to adaptive output feedback control typically make use 
of state estimation, and therefore require that the dimension of the plant is known. 
Some approaches further restrict the output to have full relative degree, or restrict the 
uncertainties in the plant to be an unknown function of the output variables. It would 
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be desirable to remove all these restrictions by adopting a direct output feedback 
approach that does not rely on state estimation. One of the immediate consequences 
of such an approach would be that the dimension of the controlled plant need not be 
known. Consequently, the resulting system would be applicable to plants having both 
parametric uncertainty and unmodeled dynamics. Furthermore, it would be desirable 
to produce a control system that is not only robust to unmodeled dynamics, but also 
learns to interact with and control these dynamics. 

Output feedback control of full relative degree systems was introduced by 
Esfandiari and Khahl, 1992, "Output feedback stabiHzation of fully linearizable 
systems," InternationalJournal of Control, 56(5):1007-1037. In their publication the 
authors formulated a control methodology that mvolves a high gain observer for the 
reconstruction of the unavailable states. A solution to the output feedback stabilization 
problem for systems in which nonlinearities depend only upon the available 
measurement, was given by Praly, L. and Jiang, Z. (1993),- "Stabilization by output 
feedback for systems with iss inverse dynamics," System & Control Letters, 21:19- 
33. Krstic, M., Kanellakopoulos, L, and Kokotovic, P. (1995;, Nonlinear and 
Adaptive Control Design, John Wiley & Sons, Inc. New York and Marino, R. and 
Tomei, P. (1995). Nonlinear Control Design: Geometric, Adaptive, & Robust. 
Prentice Hall, Inc., presented backstepping-based approaches to adaptive output 
feedback control of uncertain systems, linear with respect to imknown parameters. An 
extension of these methods due to Jiang can be foimd in Jiang, Z. (1999), A combined 
backstepping and small-gain approach to adaptive output feedback control. 
Automatica, 35:1131-1139. 

For adaptive observer design, the condition of linear dependence upon 
unknown parameters has been relaxed by introducing a neural network (NN) in the 
observer structure of Kim, Y. and Lewis, F. (1998), High Level Feedback Control 
with Neural Networks, World Scientific, NJ. Adaptive output feedback control using a 
high gain observer and radial basis function neural networks (NNs) has also been 
proposed by Seshagiri, S. and KhaUl, H. (2000), "Output feedback control of 
nonlinear systems using {RBF} neural networks," IEEE Transactions on Neural 
Networks, ll(l):69-79 for nonhnear systems, represented by input-output models. 



Another method that involves design of an adaptive observer using function 
approximators and backstepping control can be found in Choi, J. and Farrell, J. 
(2000), "Observer-based backstepping control using on-line approximation," 
Proceedings of the American Control Conference, pages 3646-3650. However, this 
result is limited to systems that can be transformed to output feedback form, i.e., in 
which nonlinearities depend upon measurement only. 

The state estimation based adaptive output feedback control design procedure 
in the Kim and Lewis 1998 pubhcation is developed for systems of the form: 

x=f(x) + g(x)^c (1) 
y = X dim x = dim y = dim u, (2) 

which implies that the relative degree of y is 2. In Hovakimyan, N., Nardi, F., Calise, 
A., and Lee, H. (1999), "Adaptive output feedback control "of ^class of nonlinear 
systems using neural networks," InternationalJoumal of Control that methodology is 
extended to fiill vector relative degree MIMO systems, non-affine in control, 
assuming each of the outputs has relative degree less or equal to 2: 

x=f(x,^c) (3) 
y = h(x) dim y = dim u < dim x. (4) 

These restrictions are related to the form of the observer used in the design procedure. 
Constructing a suitable observer for a highly nonlinear and imcertain plant is not an 
obvious task in general. Therefore, a solution to adaptive output feedback control 
problem that avoids state estimation is highly desirable. 

Summary of the Invention 
The adaptive control system (ACS) and method of this invention uses direct 
adaptive output feedback to control a plant. The system can comprise a linear 
controller (LC) and an adaptive element (AE). The linear controller can be used as a 
dynamic compensator to stabilize a model of the plant, and provide output regulation. 



The adaptive element can compensate for disturbances, and modeling error resulting 
from approximation in modeling of the plant. The adaptive element can comprise a 
neural network (NN). The adaptive element can receive a signal from the linear 
controller used to adapt its NN's weights. The input vector to the NN can comprise 
current and/or past plant output signals together with other available signals. The past 
plant output signal(s) can be used as inputs to the NN to ensure boundedness of the 
adaptive element in controlling the plant. The adaptive control system can comprise 
an error conditioning element having a low-pass filter designed to satisfy a strictly 
positive real (SPR) condition of a transfer fimction associated with Lyapunov stability 
analysis of the control system. The stability analysis can be used to construct the NN 
adaptation law using only the plant output signal(s) and other available signals as 
inputs to the NN, and to ensure boimdedness of error signal(s) of the closed-loop 
adaptive confrol system. Apparatuses forming components of the ACS are also 
disclosed. 

A method of the invention comprises generating at least one control signal 5c 
to regulate a plant output signal y by feedback of the plant output signal y, and 
optionally other sensed variables related to the state of the plant, in which 3; is a 
function of the plant state having known but unrestricted relative degree r. The control 
signal 5c can be generated so as to control the plant based on an approximate 
dynamic model, and so as to control the plant in the presence of unmodeled dynamics 
in the plant based on an adaptive confrol technique. The adaptive confrol technique 
can be implemented with a nexzral network. Related methods are also disclosed. 

These together with other objects and advantages, which will become 
subsequently apparent, reside in the details of construction and operation of the 
invented system, methods, and apparatuses as more fully hereinafter described and 
claimed, reference being made to the accompanying drawings, forming a part hereof, 
wherein like numerals refer to like parts throughout the several views. 

Brief Description of the Drawings 

Fig. 1 is a general block diagram of an adaptive control system for confroUing 
a plant based on a plant output signal 7 in accordance with the invention; 



Fig. 2 is a schematic diagram of the adaptive control system reduced to 
elements relevant to tracking error signal analysis; 

Fig. 3 is a relatively detailed view of a linear dynamic compensator of the 
5 adaptive control system; 

Fig. 4 is a relatively detailed view of an adaptive element; 
Fig. 5 is a block diagram of a processor-based adaptive control system using 
direct output feedback; 

Fig. 6 is a flowchart of a general method of the invention; 
10 Fig. 7 is a graph of commanded output signal (broken line) and plant output 

signal y (sohd hne) versus time for a control system without unmodelled dynamics 
using only a linear compensator to control a plant; 

Fig. 8A is a graph of commanded output signal y^ (broken line) and plant 
output signal y (solid line) versus time for a control system without unmodelled 
15 dynamics and with the adaptive element; 

Fig. SB is a graph of adaptive control signal v^j and inversion error signal 
A for a control system without unmodelled dynamics and with the adaptive element; 

Fig. 9 is a graph of commanded output signal jf^. (broken line) and plant output 
signal y (solid line) versus time for a control system with unmodelled djmamics and 
20 with the adaptive element; 

Fig. IDA is a graph of commanded output signal y^ (broken line) and plant 
output signal y (solid line) versus time for a control system with unmodelled 
dynamics, and with the adaptive element; and with the linear controller. 

Fig. 1 OB is a graph of commanded output signal j;^ (heavy line), plant output 
25 signal y (line with relatively moderate oscillations), and connection weights W (line 
with relatively heavy oscillations) versus time for a control system with unmodelled 
dynamics and with an adaptive neural network element and linear controller. 

Detailed Description of the Invention 
As used herein, the following terms have the following definitions: 
30 "Actuator" can be virtually any device capable of affecting the state of a plant 

to control one or more degrees of freedom thereof Such actuator can be a motor, 
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motor-driven screw, a hydraulic cylinder, a pump or valve controlling a stream of air, 
a thermal heater, a compressor or suction generator, or other device. 

"Adaptive control system" means a control system having the capability to 
5 adapt to changes in a controlled plant or its environment over time. 

"And/or" means either one or both of the elements before and after this term. 
Hence, "A and/or B" means "A" or "B" or "A and B". 

"Direct output feedback" refers to a control system, apparatus or method that 
employs feedback of an "output" that is a function of the full state(s) existing in a 
10 plant. "Direct" refers to the fact that no state estimation is used to estimate plant states 
not present in or not derivable from the "output". 

"Operator" can be a human or computer, that receives and input and generates 
and output based on the current and past history of the input., for example, senses a 
plant output using a plant output signal, and generates a commanded state signal to 
15 control the plant. - ^ 

"Memory" can be a random-access memory (RAM), read-only memory 
(ROM), erasable read-only programmable memory (EPROM), or other memory 
device capable of storing a control program and data executable by a processor. 

"Plant" refers to a system controlled by a control system. For example, the 
20 plant can be an aircraft, spacecraft, space-laimch vehicle, satellite, missile, guided 
munition, automobile, or other vehicle. The plant can also be a robot, or a pointing or 
orientation system such as a satellite orientation system to orient power-generation 
panels, a transceiver, or a docking mechanism. Such plant can also be a braking 
system, an engine, a transmission, or an active suspension, or other vehicle subsystem. 
25 The plant can be a manufacturing facihty or a power generation facility. In general, 
the plant could be virtually any controllable system. 

"Processor" can be a microprocessor such as a Xeon® or Pentium® brand 
microprocessor produced by Intel® Corporation, an Athlon® brand microprocessor 
commercially available from AMD® Corporation, Sunnyvale, California, which can 
30 operate at one (1) megahertz or more, a microcontroller, a field programmable gate 
array ("FPGA"), a programmable logic array ("PLA"), a prograiiuned array logic 
("PAL"), or other type of data processing or computing device. 
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"Relative degree" applies to a regulated variable (such as plant output signal y) 
and corresponds to the number of times the variable must be differentiated with 
respect to time before an explicit dependence on the control variable (such as the 
5 command control signal 6J is revealed. 

"Sensor" can be virtually any device(s) for sensing a degree of freedom of a 
plant's state, whether alone or in combination with one or more other sensors. The 
sensor can be virtually any device suitable for sensing information regarding a plant's 
state. For example, the sensor could be a gyroscope for detecting orientation of a 
10 vehicle such as an aircraft, i.e., pitch or roll attitudes or side slip. The sensor can also 
be a temperature or pressure sensor, a position, velocity, or inertial sensor. 

"(s)" means one or more of the thing meant by the word preceding "(s)". Thus, 
basis function(s) means one or more basis functions. 

"State" refers to a property of a plant to be controlled which is sufficient to 
15 completely define the condition of the plant at any time instant. For example, 
elements of the state can be a position, velocity, acceleration, mass, energy, 
temperature, pressure, volume, etc. of an object associated with a plant that is to be 
controlled. 

"State feedback" pertains to a simation in which the entire state of the plant 
20 can be sensed and used to control the plant through feedback. 

"Strictly positive real" is a property that pertains to the transfer fimction of a 
linear time-invariant system. The transfer function, G(s), is a ratio of polynomials in 
the variable, 's', which is a complex variable having a real and imaginary part. Let s = 
a + jb, were a is the real part and b is the imaginary part. Then the transfer function is 
25 called 'Strictly Positive Real' if the following two conditions are satisfied: 

1) G(s) is asymptotically stable (all the poles of G(s) have real parts < 0); and 

2) G(jb) > 0 for all values of the real variable 'b'. This definition can be found in 
Khahl, H.K., "Nonlinear Systems, Second Edition, Prentice-Hall, 1996, p. 404. 

"Variable" refers to any signal that can be changed independently of the plant 
30 states, such as the control variable, or that dependent upon time either directly, or 
indirectly because it depends upon plant states that are time varying, such as the 
output variable. 
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1. General Description of Adaptive Control System and Method 

As shown in Fig. 1, an adaptive control system (ACS) 10 can be used to 
5 control a plant 12 using 'direct output feedback' as opposed to 'state feedback.' In 
contrast, the disclosed adaptive control system 10 uses 'direct output feedback' in 
which a function of the full plant state, as opposed to all plant states or estimates 
thereof, is used for feedback control of the plant 12. The adaptive control system 10 of 
this invention is therefore highly useful in the control of plants, especially non-linear 

10 plants in which the full state of the plant cannot be sensed through practical or 
economically-feasible devices or techniques, particularly if the plant contains 
unmodeled or unknown dynamics. 

In Fig. 1, the ACS 10 comprises a linear controller (LC) 14, stable adaptive 
element (AE) 16, and model inversion unit (MIU) 18. The ACS 10 can further 

15 comprise error signal generator (ESG) 20 and summing unit 22_;^The ACS 10 can 
fiirther comprise an operator 20, operator interface unit 22, and command filter unit 
24. These elements permit the operator 22, which can be human br machine, to 
interact with ACS 10 to control the plant 18. The plant 12 comprises a system 30 to be 
controlled. In general, the system 30 is a non-linear system, although it can be linear. 

20 The linear controller 14 is implemented to affect approximate control of the plant, 
optionally using linear control. The AE 16 serves to implement adaptive control of 
nonlinearities of the plant 12 that are not compensated by the linear controller 14. The 
combined effect of the LC 14 and AE 16 is used to control the plant 12. 

The ACS 10 is now described in further detail. The operator interface unit 26 

25 is coupled to receive the plant output signal y which, as previously described, is a 
function of the full state of plant 12 having known but unrestricted relative degree r in 
which r is the number of times the plant output signal 3; must be differentiated in order 
for the corresponding control variable, 6<,, to become expHcit. The operator interface 
unit 26 can be an electronic interface between an ACS bus and the operator 24 if 

30 implemented as a processor, for example, or can be a display, gauge, meter, light, or 
other indicator if the operator 24 is human. The operator 24 generates command 
signals based on the plant output signal y from the operator interface unit 26. The 



operator 24 generates a command action or command signal based on the plant output 
signal y. The operator 24 supplies the command action or command signal to the 
command filter unit 28 that generates filtered commanded signals and y^""^ based 
5 thereon. The integer r denotes the relative degree of the regulated variable, and signal 
y^J^ denotes the derivative with respect to time of the filtered commanded signal 
y^ . The command filter unit 28 is coupled to supply the filtered commanded signal 
y^ to the error signal generator 20. The error signal generator 20 is also coupled to 
receive the plant output signal y. Based on the filtered commanded signal y^ and the 

10 plant output signal y, the error signal generator 20 generates a tracking error signal 5^ • 
The error signal generator 20 is coupled to supply the tracking error signal y to the 
linear controller 14. 

The linear controller 14 generates a pseudo-control component signal v^^ 
based on the tracking error signal y by operating on such error sigaal with a summing 

15 node, feedback network, and gain multipliers implementing the transfer fimction 
N^q(s) / D^q(s). The Unear controller 14 also generates a transformed signal y^^ 
based on the tracking error signal y by operating on such error signal with a siramiing 
node, one or more integrators, and a feedback network firom output terminal(s) of the 
integrator(s) that have gain multiplier(s) implementing the transfer function Nad(s) / 

20 D(ic(s). The linear controller 14 generates the transformed signal y^^ so that the 
transfer function fi-om the transformed signal y^ to the adaptive control signal Vad is 
strictly positive real (SPR). The linear controller 14 is coupled to supply the 
transformed signal y^^ to the AE 16. 

More specifically, the error conditioning element 38 of the AE 16 is coupled to 

25 receive the transformed signal y^^ . The error conditioning element 38 is also coupled 
to receive basis function(s) (j) and generates the training signal 8 based on the signal 
y^^ and the basis function(s). The error conditioning element 38 can generate the 
training signal 5 by filtering the basis function(s) (j) and multiplying the resulting 
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signal ())f by the transformed signal y^^ . The error conditioning element 38 is coupled 
to supply the training signal 5 to the neural network adaptive element (NNAE) 36. 

The NNAE 36 uses the training signal 6 to adjust connection weights W of its 
5 neural network to adapt to plant dynamics that are uimiodeled and therefore not 
adapted to by the LC 14. The delay element 40 of the AE 16 is coupled to receive the 
pseudo-control signal v and the plant output signal y and is coupled to supply non- 
delayed signals v, y, and delayed versions v^, yd thereof generated by the delay 
element 40 based on the signals v, y to the NNAE 36. The delayed signals yd are 

1 0 delayed relative to the plant output signal y by time delay increments dXo{n - \) ■ d,n 
being the number of the full plant states, although in practice fewer or more such 
delays may be used. The delayed signal(s) vd are delayed relative to the pseudo- 
control signal V by time delay increments d to {n - r - \) • d , r being the relative 
degree. The use of these delayed signals assures that the weight coefficient(s), W, and 

15 error signal(s) y are bounded so that the ACS 10 is stable. 

Based on the connection weight(s) W determined by training signal 6, and the 
pseudo-control signal v and delayed version(s) thereof and/or the plant output 
signal y and delayed version(s) yd thereof supplied as inputs to the NNAE 36, the 
NNAE generates the adaptive control signal v^d- The NNAE 36 is coupled to supply 

20 the adaptive control signal Vad to the summing unit 22. The summing unit 22 is also 
coupled to receive the pseudo-control component signal \dc from the linear controller 
14 and the r-th time derivative of the commanded state signal y^^^ from the command 
filter unit 28. Based on the signals y^J^ , vdc > ^ad > the svmiming unit 22 generates the 
pseudo-control signal v. The summing unit 22 is coupled to supply the pseudo-control 

25 signal v to the model inversion \mit 18. 

The model inversion unit 18 is also coupled to receive the plant output signal 
y. The model inversion unit 18 generates a command control signal 5^ based on the 
pseudo-control signal v and the plant output signal y. More specifically, the model 
inversion unit 18 subjects the signals v, to a function that inverts the linear control 

30 model implemented by the linear controller 14, to produce the command control 



signal 8^. The model inversion unit 18 is coupled to supply the resulting command 
control signal S^to the actuator(s) 32 of the plant 12. The actuator(s) 32 are coupled to 
or associated with the controlled nonlinear system 30 so that the control the state(s) of 
5 such system, based on the command control signal 5,.. The sensor(s) 34 are coupled or 
associated with the controlled nonlinear system 30, and generates the plant output 
signal y that is a function of the full plant state(s) of the controlled nonhnear system 
30. 

Model inversion in the unit 18 can be performed in the following way. For the 
10 scalar case (p=l) if the regulated output, y, has relative degree r, then the output 
equation can be differentiated r times with respect to time until the control appears 
explicitly. In this case, we assume that we have a model of this derivative 

y(^)=h,(y,5e) = v . (5) 

15 

Thus the model inversion of the xmit 18 is defined by 

S,-^h;\y,v). (6) 

20 2. Specific Description of Adaptive Control System and Method 

Let the dynamics of an observable nonlinear single-input-single-output (SISO) 
system be given by the following equations: 

x=f(x,^e), y = h(x) (7) 

25 

where x e Q c 91" is the state of the system, 5^ > Y ^ ^ ^re the system input (control) 
and output (measurement) signals, respectively, and f (•,•), h(-) e C" are unknown 
functions. Moreover, n need not be known. 

To ensure proper operation, the following assumption is made in the 
30 implementation of the ACS 10. 
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Assumption 1. The dynamical system of Eq. (7) satisfies the output feedback 
linearization conditions Isidori, A. (1995), Nonlinear Control Systems, Springer- 
Verlag, Inc. with relative degree r, i.e., 

5 

y«=hr(x,5,). (8) 

Here, h^ A-^,such that = 0 for 0 < i < r and-^^ 9^ 0. 
~ dt' 5u 5u 

10 This disclosure addresses the design of an output feedback control law that utilizes the 
available measurement y(t), to obtain system output tracking of a bounded trajectory 
yc(t) that is assumed to be r-times differentiable, i.e., y^ g C . The difference between 
unknown dynamics function h^ and its estunateh^, or the inodeling error, is mapped 
using a NN. This mapping has to be based on measured input and^butput data only. 

15 To this end, the universal approximation property of neural networks and the 
observability of the system are utilized to construct this mapping on-line using 
measured input/output time histories. These various features of the proposed control 
design scheme are presented in the next section. 

3 Controller Design 

20 3.1 Feedback Linearization 

Feedback Unearization is performed by introducing the transformation 

v = h,(y,6c), (9) 

25 where v is commonly referred to as a pseudo control signal, and hj.(y,6c) is the best 
available approximation of hi.(x,5c). Then, the system dynamics can be expressed 
as 

y<'^=v + A', (10) 

30 where 
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A' = A'(x, 5, ) = h, (X, 6c ) - hr (y> 5c ) . 



(11) 



Using this transformation, Eq. (10) represents the dynamic relation of r integrators 
5 between the pseudo-control signal v and the plant output signal y, where the error A' 
acts as a disturbance signal. Assuming that the plant output signal y is required to 
track a known bounded input command signal y^, the pseudo-control signal v is 
chosen to have the form 

10 v = y<;>+v,,-v,„ (12) 

where v^^ is the output of a stabilizing linear dynamic compensator for the linearized 
dynamics in Eq. (10) with A' = 0, and v^^ is the adaptive control signal designed to 
approximately cancel A' . The r-th derivative of the input signal, y^'^ , is introduced as 
15 a feedforward term to condition the error dynamics. This derivative can be easily 
obtained if the tracking (or command) signal is generated using an r-th (or higher) 
order reference model forced by an external input. The reference model serves to 
define the desired response of the closed loop system. The input to the dynamic 
compensator is the tracking error, which is defined by 

20 

y = yc-y- (i3) 

It is important to point out that the model approximation fimction hr(-,0 should be 
defined so that it is invertible with respect to u, allowing the actual control input to be 
25 computed by 

8c=hr^(y,v). (14) 
Clearly, the accuracy of the approximation h^(x,h;:'(y,v))~ v is governed by 

30 
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A'(x,5e) = A(x,y,v) = (x,h7i(y, v))-v . (15) 

5 

From Eq. (11) and Eq. (15), notice that A depends on v^^ through v, whereas v^^ has to 
be designed to cancel A. The following assumption is introduced to guarantee 
existence and uniqueness of a solution for v^: 

10 Assumption 2. The map v^^ i-> A is a contraction over the entire input domain of 
interest. 



Using Eq. (11), the condition in Assumption 2 imphes: 



dA 



a(hr-hr)38c ^ 

du dv dv„A 



a(h,-h,)a8e 
au ahr 



(16) 



which can be re-written in the following way: 



ah J, /55c 



(17) 



The condition (17) is equivalent to the following two conditions 



(1) sgn(ahr/a5c)=sgn(ahr/a5c) 

(2) \dh,/d8J>\dh^/d8c\/2>0. 



25 The first condition means that control reversal is not permitted, and the second 
condition places a lower bound on the estimate of the control effectiveness in (14). 
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3.2 Control System Architecture 

Based on the above description, the overall control system architecture is 
presented in Fig. 1. The central components of the system are: (a) the model 
inversion/linearization unit 18 implementing block h^ ^(y,5c) , (b) the adaptive neural 
network based element 16 is designed to minimize the effect of A, and (c) the linear 
dynamic compensator of the linear controller 14. The input into the ACS 10 is the 
reference command trackmg signal y<, and its r-th derivative y^/' , generated by, e.g., a 
reference model forced by an external input. 

It is important to note the two output signals (Vj,,y,j) of the linear 
compensator. The pseudo-control component signal, v^^, is designed to stabilize the 
linearized system, as described earlier. The transformed signal, y^^, is a linear 
combination of the compensator states and its input, i.e., the tracking error signal y . 
This signal is generated to ensure an implementable error signal 5 that is used to adapt 
the NN weights W of the NNAE 36. 

3.3 Tracking Error Signal Analysis 

The analysis presented in this subsection is carried out to facihtate the design 
of the NNAE 36 and the second output signal y^^ of the linear dynamic compensator 
14. To formulate the overall tracking error dynamics of the controlled system, the 
specific choice of the pseudo-control signal v is given by Eq. (12) is substituted into 
Eq. (16), leadmgto 

y(r) ^ y(r) + _ + ^ ^ (18) 

or alternatively 

5?w=-v,,+v„,-A. (19) 

These error dynamics are depicted schematically in Fig 2. More specifically, under 
tracking error dynamics analysis, the ACS 10 reduces to a summing node 42, an 
integrator 44, and a linear dynamic compensator (LDC) 46. The summing node 42 is 
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coupled to receive the signal v^j - A and the pseudo-control component signal v^c- The 
summing node 42 subtracts the signal v^,; from the signal v^^ - A to generate the signal 
y . The r-th degree integrator 44 integrates the signal >^ to produce the signal y . 
5 The LDC 46 is coupled to receive the signal y from the integrator 44. Based on the 
signal y , the LDC 46 generates the signal v^^ that is fedback to the summing node 42. 
The LDC 46 also generates the signal y^^ based on the signal y. The LDC 46 
implements a transfer function that is SPR to map the signal y to the signal v^^ - A to 
ensure stability of the ADC 10. 

10 

The single-input two-output transfer matrix of the linear dynamic compensator is 
denoted by 




15 

where s represents the complex Laplace variable. The LDC 46 can comprise transfer 
function elements 48, 50. The transfer function element 48 can be used to implement 
the transfer function N<i,(s)/D^(s) mapping the signal y to the signal v^c. The 
transfer function element 50 can be used to implement the transfer function 
20 (s) I (s) map the signal y to the signal y^^ . Further details regarding the LDC 

46 are described below. 

Assumption 3. The linearized system in Fig. 2 is stabilized using a stable linear 
dynamic compensator 46, i.e., the roots of the denominator polynomial D(}c(s) are 
25 located in the open left half plane of the complex plane s. 

Since the linearized system dynamics, and hence the error dynamics, consist of r pure 
integrators, this assumption introduces only a very mild restriction on the design. 
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Based on the compensator defined in Eq. (20), the closed loop transfer function of the 
system depicted in Fig. 2 is given by: 



5 



■(v,,-A)(s)AG(s)(v,,-A)(s). 



(21) 



s^D,,(s) + N,,(s) 



Analyzing the denominator of Eq. (21), the Routh-Hurwitz stability criterion impHes 
that a necessary condition for closed loop system stability is that the degree of the 
compensator numerator, N^{s) , and hence of its denominator, D^^(s) , should be at 
10 least (r - 1), i.e.. 



which can be carried out using any linear control design technique (classical, pole 
placement, optimal LQ, etc.), with the constraint of assumption 3. The numerator 
20 N^dCs) > associated with the output y^^, does not affect the stabihty of the error system 
of Fig. 2. 

3.4 Neural Network Based Approximation 

The term "artificial neural network" has come to mean any architecture that 
25 has massively parallel interconnections of simple "neural" processors. Given x e R'^' , 
a three layer-layer NN has an output given by: 



qAdeg(D,,(s))> deg(N,,(s)) > r - 1 . 



(22) 



This dictates the design of the linear dynamic compensator 



15 




(23) 




(24) 
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where ((>(.) is the activation fiinction, are the first-to-second layer interconnection 
weights, and Wjj are the second-to-third layer interconnection weights. 0^ and 9^ 
are bias terms. Such an architecture is known to be a universal approximator of 
continuous nonlinearities with squashing activation functions. See Funahashi, K. 
(1989), On the approximate reaHzation of continuous mappings by neural networks. 
Neural Networks, 2:183-192; Homik, K., Stinchcombe, M., and White, H. (1989), 
Multilayer feedforward networks are universal approximators. Neural Networks, 
2:359-366. 

Linearly parameterized neural networks 

y = W^0(x) (25) 

are universal approximators as well, if vector function 0(-) can be selected as a basis 
over the domain of approximation. Then a general function f(x) e , 
x e D iR" can be written as 

f(x) = W^0(x) + 8(x), (26) 

where £(x) is the functional reconstruction error. In general, given a constant real 
number 8* >0,f(x) is within s* range of the NN, if there exist constant weights W, 
such that for all x e SR" Eq.(20) holds with ||8|| < 8* . 

Definition 1. The fimctional range of NNAE 36 is dense over a compact domain 
X e D , if for any f(-) € C'' and s* there exists a finite set of bounded weights W, such 
that Eq. (26) holds with |s|| < £* . 

Various publications show that the functional range of NN in Eq. (25) is dense 
for different activation fimctions 0(-). See Cybenko, G. (1989) publication. 
Approximation by superpositions of sigmoidal function. Mathematics of Control, 
Signals, Systems, 2(4):303-314; Park, J. and Sandberg, I. (1991), Universal 
approximation using radial basis function networks, Neural Computation, 3:246-257; 
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Sanner, R. and Slotine, J. (1992), Gaussian networks for direct adaptive control, IEEE 
Transactions on Neural Networks, 3(6):837-864. 

The following theorem extends these results to map the unknown dynamics of 
5 an observable plant from available input/output history. 

Theorem 1. Given s* >0, there exists a set of boimded weights W, such that 
A(x,y,v) , associated with the system (l)-(5), can be approximated over a compact 
domain D <= Q x R by a linearly parameterized neural network 

10 A = W"0(Ti) + g(Ti), ||s||<s* (27) 

using the input vector 

Tl(t) = [l vj(t) yliX)], (28) 

15 where 

vl(t) = [v(t) v(t-d) - v(t-(n,-r-l)d) f 



yl(t) = [y(t) y(t-d) ••• y(t - (n^ - l)d) f 



20 with ni > n and d > 0 , provided there exists a suitable basis of activation functions 
0(-) on the compact domain D. 

The output of the adaptive element 16 in Fig. 1 is designed as 



v,, = WV^), (29) 

25 

where W are the estimates of the weights. Eq. (29) will always have at least one fixed- 
point solution, so long as (i(-) is made up of bounded basis functions. 



30 
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3.5 Construction of SPR Transfer Function 

As discussed earlier, the second output of the linear dynamic compensator 46, 
, will be used to construct the rule for adapting W in Eq. (29). Using Eqs. (27) and 
5 (29) in Eq. (21) implies: 



where W = W - W is the weight error. As will be seen in the next section, for the NN 
10 adaptation rule to be reaUzable, i.e. dependent on available data only, the transfer 
function G(s) must be strictly positive real (SPR). However, the relative degree of 
G(s) is at least r. When the relative degree of G(s) is one, it can be made SPR by a 
proper construction of N,j(s). If r > 1 , G(s) cannot be SPR through this technique 
alone. 

15 To achieve SPR in the r >1 case, following the Kim and Lewis, 1998 

publication, a stable low pass filter T~^(s) is introduced in Eq. (30) as:' 



20 where 0j.and Sf are the signals 0 and £, respectively, after being filtered 
through T"^(s) , and 5^(s) is the "so-called" mismatch term given by 




(30) 




(31) 



5i„(s) = T"^ 



■^(s)(w'^0)-W'^0f 



(32) 
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that can be bounded as 



||8^(t)||<c||w||p^ c>0. 



(33) 
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The numerator of the transfer function G(s)T(s) = G(s) in Eq. (31) is _T(s)N,d(s) . 
The polynomial T(s) is Hurwitz, but otherwise can be freely chosen, along with the 
numerator polynomial N33(s)of Eq. (21) that defines the output y.^ • Hence, the 
5 numerator polynomial (or the zeros) of G(s) can be freely chosen to make it SPR. 
Two approaches can be utilized in constructing T(s) and N,<j(s) to makeG(s) SPR. 
To avoid an unreaUzable feedthrough, G(s) will be assigned (r + q - 1) zeros, thus 
making it relative degree one. 



10 Zero placement approach: Since G(s) is a stable transfer ftmction, its zeros can be 
easily placed to make it SPR, e.g., by interlacing them with its poles. From Bode plot 
analysis it is easy to conclude that such a pole-zero pattern will ensure a phase shift in 
the range of ± 90° . 



LKY approach: Assume that 



_ b^s"-' +b,sP-^+--- + b„ 

G(s) = -4 — hr, — \ — - (34) 



where p = r + q is the number of the closed loop poles. The confroller canonical state 
space reahzation of this transfer function is given by 

20 z = A,iZ + B,,(w'^0f + 5-Sf) 



yad=CoiZ, (35) 

where 





-aj - 


a, ••• -a; 




"r 




1 


0 ••• 0 




0 




0 




Bel = 


0 




0 


•■•0 1 0 




0 



21 



C., = [ \ b, - bJ 



G(s) is SPR if and only if it complies with the Lefschetz-Kalman-Yakubovitz (LKY) 
5 Lemma, loannou, P.A. and Sun, J. (1996), Robust Adaptive Control, Prentice Hall, 
Inc., p. 129, i.e., there exists Q)-0 such that the solution P of 



A^,P + PA,,=-Q (36) 

is positive definite and 

PB„=Cj,. (37) 



G(s) can be constracted utilizing the LKY condition as follows: 

1 5 a) Choose Q ^ 0 and solve Eq. (36) for P :^ 0 . 

b) Using Eq. (37), compute C^, , which in this canonical form is 

simply the first column of P. From Eqs. (33) and (34), the elements of 
C^, are also the coefficients of the numerator polynomial of G(s) . Since 
G(s) is SPR, it is guaranteed that this numerator is Hvirwitz. 

20 c) Solve the numerator polynomial for its roots. 

From the zeros obtained by either of the above methods, choose (r - 1) of these to 
construct T(s), while the remaining q zeros makeup N3a(s). The fact that the 
numerator of G(s) is Hurwitz ensures also that T(s) and N^j(s) are individually 
25 Hurwitz. There is freedom m scaling T(s) and N^d(s), which could be utilized to 
normalize the maximum gain of T~'(s) . 

To summarize, Nj^(s)/D^^(s) is designed to stabilize the linearized system 
dynamics, while N^d(s)is constructed to meet the SPR condition needed for a 
realizable implementation. 
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Neural Network Adaptation Rule 



5 As is evident from Eq. (31), the filter T '(s) should operate on ail the components of 
the NN vector 0 . All these filters can be cast in one state space realization: 

Zj. = Af Zf + Bf 0 

(38) 

10 0f=CfZf, 

where the diagonal blocks of the state space matrices (A^ ,Bf ,Cf) are constructed 
from a state space realization of the filter T"^(s) . Since the filter is stable, 3 Pf ;^ 0 , 
satisfying 

15 

A^P,+P,A,=-Q, (39) 

for any positive definite :^ 0 

20 The signals 0f are used in the following NN W weight adaptation rule 

dW / dt = -F[yad0f + A, w Wj, (40) 

where F 0 and A,^ > 0 are the adaptation gains. In the next section it is proven that 
25 this adaptation rule ensures boundedness of the system error signals and the network 
weights. The NNAE 16 of Fig. 1 is depicted in more detail in Fig. 4. 

Fig. 4 is an exemplary embodiment of the adaptive element 16 provided by 
way of example and not limitation as to possible implementations of the NNAE 16. 
The time delay element 40 comprises one or more time-delay (TDL) vmits 521-52^,1.^., 
30 coupled to receive the pseudo-control signal v, and TDL units 54i-54„n coupled to 
receive the plant output signal j;. The TDL units 52,-52„,.r.i, 54i-54„,.i generate delayed 
23 



versions v^, of the signals v, y , and are coupled to supply these delayed signals 
Vd, y^ as well as undelayed signals v, y, to the NNAE 36. The neural network (NN) 
64 of the NNAE 36 multiplies the signals v, v^, y, y^ by respective weight data V and 
transmits the resulting signals to respective basis functions <})(•) 56„ 562,..., 56^1 • 
The basis functions ())(•) 56^, 562,..., 56^1 are coupled to receive V-weighted signals 
V, v<j, y, y^ and generate respective signals based thereon. The generated signals are 
multiplied by respective weight data W and summed at respective summation nodes 
58i, 582, . . ., 58^2- The NNAE 36 is coupled to supply the resulting summed signals as 
the vector signal v.^ to the summmg node 22 of Fig. 1 for generation of the pseudo- 
control signal V. 

To ensure boundedness of the basis functions <])(•) 56i, 563,..., 56^, and neural 
network weights V,W, the NNAE 36 is coupled to supply the basis functions ([)(•) 56„ 
562,..., 56ni as signals to the error conditioning element 38. The error conditioning 
element 38 comprises a filter 60 and a multiplier 62. The filter 60 operates on the 
basis functions 56„ 562,... ,56^,1 with a fdtering transfer function T'^(s) as previously 
described with respect to Equation (32) to generate filtered basis functions <j)f (•)• The 
filter 60 is coupled to supply the filtered basis functions (j)f (•) to the multiplier 62. The 
multiplier 62 is also coupled to receive the transformed signal y^^. The multiplier 62 
generates the signal 5 that is a vector product of the signals (j)f (•), y^^. The multiplier 62 
is coupled to supply the signal 6 to the NNAE 36. Based on the signal 5, the NNAE 
36 adjusts the weight data W to adapt the NNAE 36 to generate the pseudo-control 
signal so as to compensate for error A associated with the command control signal 5^. 
4. Boundedness Statement 

The following theorem establishes sufficient conditions for boimdedness of the 
error signals and neural network weights in the proposed closed-loop adaptive output 
feedback architecture. 

Theorem 2. Subject to assumptions 1-3, the error signals of the system comprised of 
the dynamics in Eq. (7), together with the dynamics associated with the realization of 



24 



the controller in Eg. (14) and the NN adaptation rule in Eg. (40), are uniformly 
ultimately bounded, provided the following conditions hold 

5 Q.>2||C,,| X^>cV4, (41) 

where is the minimum eigenvalue of Q. 

5. Processor-Based Embodiment of Adaptive Control System using Direct 
10 Output Feedback 

Although it is possible to implement the elements 14, 16, 18, 20, and 22 of the 

ACS 10 of Fig. 1 as discrete or grouped analog or digital devices, these elements can 
alternatively be implemented in a processor-based ACS system 10. The processor- 
based system 10 includes a processor 66 and memory 69 storing data and a control 

15 program, to implement the elements 14, 16, 18, 20, and 22. More specifically, the 
control program can be implemented as software objects or modiries that perform the 
functions of the elements 14, 16, 18, 20, 22 as previously described. The data can be 
parameters such as the NN connection weights W, V and^or basis function(s) (|) that 
are updated by the processor 66, as well as temporary data and intermediate 

20 calculations, commanded state signal levels, plant output signal levels, etc. The ACS 
10 of Fig. 5 can further comprise bus 70 to which the operator interface unit 26, the 
command filter imit 28, the actuator 32, the sensor(s) 34, the processor 66, and the 
memory 68, are coupled. 

In operation, the sensor(s) 34 generate plant output signal y and supply this 

25 signal to the operator interface unit 26 via the bus 70. The operation interface unit 26 
generates a signal readable or discernible by the operator. If the operator 24 is human, 
the operator interface unit 26 can generate a display or the like based on the plant 
output state signal y. If the operator 24 is a processor or machine, the operator 
interface unit 26 can convert the plant output state signal y into a format usable by the 

30 operator. The operator 24 if human produces one or more signals through control 
actions applied to a command filter unit 28. For example, in the case of an aircraft, the 
control actions may be applied to control instruments of the aircraft. Alternatively, if 
the operator 24 is a machine, the command signal produced by the operator can be 
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applied to the command filter unit 28. The command filter vmit 28 generates the 
commanded output signal and the rth derivative of the commanded output signal 
y}^\ The command filter unit 28 suppUes the signals y^, y^^^ to the processor 66 or to 
5 the memory 68 at a location accessible to the processor 66. The sensor(s) 34 can 
supply the plant output signal y directly to the processor 66, or to the memory 68 at a 
location accessible to the processor 66 via the bus 70. The processor 66 performs the 
functions of the elements 14, 16, 18, 20, 22 to generate a command control signal 5^. 
The processor 66 is coupled to supply the command control signal 5^ to the actuator(s) 

10 32 via the bus 70. The actuator(s) 32 perform control of the plant 12 in a manner that 
can affect the plant state(s). The sensor(s) 34 sense and generate the plant output 
signal y for the next control cycle. Processing performed by the processor 66 in 
executing its control program can be repeated over successive control cycles as long 
as required to control the plant 12. 

15 6. General Method of the Invention * 

Fig. 6 is a flowchart of processing performed by the ACS 10 of Figs. 1-5. In 
step SI of Fig. 6 the method begins. In step S2 a command control signal 5^. is 
generated by inverting an approximate model of the plant dynamics, based on a 
pseudo-control signal v and the plant output signal y. In step S3 the command control 

20 signal 5<, is supplied to control the plant. In step S4 the plant output signal y is 
generated the sensors. In step S5 a tracking error signal 5^ is generated by 
differencing corresponding signal components of the commanded output signal and 
optional derivative(s) thereof, and the plant output signal y. In step S6 a pseudo- 
control component signal v^^ is generated based on the tracking error signal y using 

25 the transfer function Nj,(s)/D^,(s) . In step S7 a transformed signal (Note change 
here) y^^ is generated based on the tracking error signal y using transfer function 
Nad(s) /DcJc(s). In step SB the rth derivative of the commanded output signal y^^^^ is 
generated. In step S9 the signal y^^ is generated to render the transfer function firom 
the signal Vad to the signal y^j strictly positive real by appropriate choice of N^dCs) 

30 . In step SIO a training signal Sis generated by filtering basis function(s) ^ and 
multiplying the filtered basis function(s) ^ by the transformed signal y^^ . In step SI 1 



connection weights W of a neural network are updated in a bounded manner using the 
training signal 8. In step S12 delayed versions of the pseudo-control signal v are 
generated. In step S13 delayed versions of the plant output signal y are generated. In 
5 step S14 the adaptive control signal Vad is generated based on the pseudo-control 
signal V, delayed versions v^f of the signal v, plant output signal;;, plant output signal 
y^, connection weights W, V, and basis function(s) ^ updated based on the training 
error signal 5. In step S15 a pseudo-control signal vis generated based on the rth 
time-derivative of the commanded output signal y^^, pseudo-control component 
10 signal v^, and adaptive control signal v^^. In step S16 the method of Fig. 6 ends. 

7. Example of Implementation of the Adaptive Control System having Direct 
Output Feedback Control 

15 The performance of the ACS 10 using output feedback is demonstrated by 

considering the following nonlinear system, consisting of a modified Van der Pol 
oscillator coupled to a lightly damped mode 



±,-^^ (42) 

20 ±2 = -2(xf - 1)X2 - X, + u (43) 

X3 = X, (44) 

X4=-X3-0.2x4-i-x, (45) 

y = Xj + X3 (46) 



25 The output y has a relative degree of r = 2. From a practical perspective, the 

system can be thought of as a second order nonlinear plant model, whose reaUzation 
consists of states x, and x^, in which the output is modeled as y = Xj. However, the 
system contains also a very lightly damped unmodeled mode, with a natural frequency 
equal to that of the linearized plant. This mode is excited by the plant displacement 

30 state (Xj) and is coupled to the measurement. 

The output signal y does not have a full relative degree in the presence of the 
unmodeled mode. The low natiu-al frequency of this mode is encompassed by the 

27 



bandwidth of the controlled system. This introduces a challenging control problem, in 
particular for methods that require the output to have a full relative degree. Moreover, 
this example is treated as if even the Van der Pol model is vmknovm, and only the fact 
that r = 2 is given. This is not an unreasonable assumption in that in many systems, 
the number of plant states and hence the value of r can be deduced from knowledge of 
the behavior of the plant. Thus, the controller design is performed assuming y = u , 
implying that in Fig. 1 the plant transfer function from the pseudo-control signal v to 
y is l/s^ . 

A first order lead-lag compensator was selected to stabilize the associated error 
dynamics. In addition, the first design approach described in Section 3.5 was used to 
satisfy the SPR condition. The resulting two outputs of the compensator are given by 




which places the closed loop poles of the error dynamics at - 3,-1 ± j . The low pass 
filter 60 T'\s) discussed in Eq. (26) was chosen as 

^"■<^> = a5b- '''' 

It is easy to verify that the transfer function G(s)T(s) of Eq. (31) is SPR. 

A Gaussian Radial Basis Function (B^BF) NN with only three neurons and a 
bias term was used in the adaptive element. The fimctional form for each RBF neuron 
was defined by 

<P,(?j) = e-(Tj-rjJ^{n-n^,)/cT\ O" = V2 , / = 1,2,3. (52) 
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The centers rici,i = 1,2,3 were randomly selected over a grid of possible values for the 
vector ri . All of the NN inputs were normalized using an estimate for their maximum 
values. The current and two delayed values for the plant output signal y and only the 
current pseudo-control signal v were used in the input vector to the neurons. The 
complete input vector consisted of these values together with a bias term, as illustrated 
in Fig. 4. Thus, there are a total of four NN weights in Eq. (29). The network gains 
were F= 50 and - I . 

In the simulation, the initial states of the system were set to X](0) = 0.5, 
x^(0)=2.5, XjCO) = X4(0) = 0. The system was commanded to follow the output of a 
second order reference model for the MIU 18, designed with a natural frequency of 
co„==l[radysec] and damping C,-2/^l2 , and driven by a square wave input command 
signal 

First, the controlled system performance is evaluated without the unmodeled 
mode dynamics, i.e., removing Eqs. (44) and (45) and setting the output y = x,. 
However, it will be recalled that the controller has been designed given only the fact 
that r = 2. Fig. 7 compares the system response without NN augmentation (solid line) 
with the reference model output (dashed line), clearly demonstrating the almost 
unstable oscillatory behavior caused by the nonUnear elements in the Van del Pol 
equation. Figs. 8A and SB show that with NN augmentation, these oscillations are 
eliminated after a period of about three seconds. This is accounted for by the 
successful identification of the model inversion error by the NN, which is also 
illustrated in Fig. SB by comparing the NN output (solid line) with the computed 
inversion error (dashed line). 

Next, the effect of the unmodelled dynamics is examined. In this case, the 
response without the NN is unstable, and therefore is not shown. The response with 
NN augmentation is presented in Fig 9. It shows only minor performance degradation 
compared to the full relative degree case of Fig. 8A. Since the unmodeled mode is 
well within the bandwidth of the control system (when viewed with v^^ = A), this 
demonstrates that the adaptive system learns to interact with the added mode to 
achieve good tracking performance. 
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In Figs. 8A, 8B, 9 the NN based adaptive controller exhibits a steady state 
tracking error. This error can be removed by introducing an additional integral 
control action when designing the linear compensator. The performance of the 
5 controller with integral action is shown in Figs. lOA and lOB. The steady state 
tracking error is zero, while the transient response behavior is only shghtly 
compromised. The bounded NN weight time histories are also depicted in this figure, 
showing that most of the weight adaptation takes place when the command reverses 
direction. 

10 8. Additional Considerations 

The stability results are semiglobal in the sense that they are local with respect 
to the domain D. If the NN universally approximates the inversion error over the 
whole space Sft"""* , then these results become global. 

The NN update laws consist of a modified gradient algorithm along with the 
15 standard a-modification term as described in the Kim and Lewis-1998 pubUcation. 
These laws have been proven to be passive in Lewis, F. (1999), Nonlinear network 
structures for feedback control, Asian Journal of Control, l(4):205-228. 

The NN learning takes place on-line, and no off-line training is required. No 
assumption on persistent excitation is required. 
20 The ultimate bound for the tracking error can be made smaller by increasing 

the linear design gains. This will result in increased interaction with unknown or 
immodeled plant dynamics. However, Theorem 2 remains vaUd so long as 
assumptions 2 and 3 hold. 

In the case of plants of unknown dimension but with known relative degree, 
25 the described methodology apphes with a sUght modification of the input vector to the 
network: a large range of input/output data should be used, i.e., Uj » n . 

9. Conclusion 

The described ACS 10 presents an adaptive output feedback control design 
procedure for nonlinear systems, that avoids state estimation. The main advantage is 
30 that the stability analysis permits systems of arbitrary but known relative degree. The 
Ml dimension of the plant and its internal dynamics may be known or poorly 
modeled. Only mild restrictions regarding observability and smoothness are imposed. 



Consequently, the result is applicable to adaptive control of nonlinear systems with 
parametric uncertainty and unmodeled dynamics. 

Any trademarks listed herein are the property of their respective owners, and 
reference herein to such trademarks is intended only to indicate the source of a 
particular product or service. 

The many features and advantages of the present invention are apparent from 
the detailed specification and it is intended by the appended claim to cover all such 
features and advantages of the described methods and apparatus which follow in the 
true scope of the invention. Further, since nimierous modifications and changes will 
readily occur to those of ordinary skill in the art, it is not desired to limit the invention 
to the exact implementation and operation illustrated and described. Accordingly, all 
suitable modifications and equivalents may be resorted to as faUing within the scope 
of the invention. 
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